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Abstract 

It has been previously shown that one can use the ME methodology (Caticha Giffin 2006) to repro- 
duce a mean field solution for a simple fluid (Tseng 2004). One could easily use the case of a simple 
ferromagnetic material as well. The drawback to the mean field approach is that one must assume that 
all atoms must all act the same. The problem becomes more tractable when the agents are only allowed 
to interact with their nearest neighbors and can be in only two possible states. The easiest case being an 
Ising model. The purpose of this paper is to illustrate the use of the ME method as an approximation 
tool. The paper show a simple case to compare with the traditional mean field approach. Then we 
show two examples that lie outside of traditional methodologies. These cases explore a ferromagnetic 
material with defects. The main result is that regardless of the case, the ME method provides good 
approximations for each case which would not otherwise be possible or at least well justified. 

1 Introduction 

In a simple ferromagntic material (single domain), the electronic spin of the individual atoms are strong 
enough to affect one and other, the so called "exchange" interaction pQ. However this effect is temperture 
dependent. When the temperture is below a certain point (the Curie tempreture) the spins tend to all 
point in the same direction due to their influence on each other. This establishes a permanant magnet as 
the individuals produce a net dipolc effect. Above this temperture, the atoms cease to have a significant 
effect on each other and the material behaves more like a paramagnetic substance. Determining this net 
dipolc effect can be difficult. First, the interactions are due to complicated quantum effects. Second, since 
a given material as a very large number of atoms, computing the net dipole effect can be difficult in two 
dimensions and completely intractable in three dimensions. Therefore approximations are made to facilitate 
computational difficulties such as using an Ising Model and or the mean field approximation. 

In 1957, Jaynes [2] showed that maximizing statistical mechanic entropy for the purpose of revealing 
how gas molecules were distributed was simply the maximizing of Shannon's information entropy [3] with 
statistical mechanical information. This idea lead to MaxEnt or his use of the Method of Maximum Entropy 
for assigning probabilities. This method has evolved to a more general method, the method of Maximum 
(relative) Entropy (ME) [1] which has the advantage of not only assigning probabilities but updating them 
when new information is given in the form of constraints on the family of allowed posteriors. One of the 
drawbacks of the MaxEnt method was the inability to include data. When data was present, one used 
Bayesian methods. The methods were combined in such a way that MaxEnt was used for assigning a prior 
for Bayesian methods, as Bayesian methods could not deal with information in the form of constraints, such 
as expected values. 

Previously it has been shown that one can use ME to reproduce a mean field solution for a simple fluid 
[5]. The purpose of this was to illustrate that in addition to updating probabilities, ME can also be used 
an an approximation tool. The purpose of our paper is to illustrate the use of the ME method as a tool for 
attaining approximations for ferromagnetic materials that lie outside the ability of traditional methods. In 
doing so we further the the previous work done and show the versitility of the method. 



*E-mail: physicsl01@gmail.com 



1 



In the second section of this paper we show the traditional methods for attaining an approximation, 
the Ising model and the mean field field approximation. The third section shows how ME can not only 
reproduce the mean field approximation, but does so using fewer assumtpions. In the fourth section we 
provide two examples that lie outside the capabilities of traditional methodologies. These cases explore a 
ferromagnetic material with defects. The main result is that regardless of the case, the ME method provides 
good approximations for each case which would not otherwise be possible or at least well justified. 



2 The Ising model and the mean field approximation 

The drawback to the mean field approach is that one must assume that all atoms must all act the same. 
Additionally, the problem is only tractable when the atoms are only allowed to interact with their nearest 
neighbors and assumed to be in only two possible states. This model is called the Ising model. 



2.1 The Ising Model 

Althought these exchange interactions are quantum related, Ernst Ising [6 suggested that one should examine 
a simple model where the atomic spins are related by simple spins of +1 and -1 or spin "up" and spin "down" . 
Further, he suggested to neglect all exchange interactions except those between nearest neighbors. Using 
these two assumptions, the exchange interaction energy can be calculated as follows: 

N,n 

H=-jJ2siSj, (1) 

where N is the number of atoms, J is the exchange interaction energy and s is the net electronic spin of the 
atom where s = {+1, —1}- The standard partition function for this system would then be written as, 

Z = J2 e-e* , (2) 
{*} 

where } = S Sl S S2 ■ ■ ■ S SN an d ft is usual thermodynamic value 1/kT where T is the temperture and 
k is the Boltzmann constant. For the one dimentional case, the partition function can easily be calculated, 

Z = 2 JV (cosh/3J) Ar - 1 w (2 cosh /3 J) N (for large N). (3) 



The resulting average energy would then be, 



H = -^S? = -NJteaih/3J . (4) 
op 



2.2 The mean field approximation 

Although the Ising model's partition function can be solved exactly, this is only true for one dimension. It fails 
for real ferromagnetic substances since it only takes into account two near neighbors. For three dimensional 
materials, each atom may have 6, 8 or even 12 close neighbors depending on the crystal geometry. This is 
where another approximation is made and it is commonly refered to as the mean field approximation [TJ [S] . 

We start with the same energy function as above (QJ but now examine only one atom (sometimes called 
the "central" atom), 

n 

Hi = -Jsi 2J Sj . (5) 
>j 

Where Hi is the energy for one atom and n is the number of nearest neighbors (ex. a face-centerd cubic 
lattice would have 12) with atoms at the edge of the material neglected. Now we assume that each neighbor 
contributes equally and average over the neighbor spins around this one atom, 

Hi = -Jsinsj , (6) 

where Si = YTj ~k an d the Boltzmann probablity for finding this atom in the s< state is 

Pi = U-e* , (7) 
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where Z\ = 2 cosh /3JriSj. Now we take the the average of for this one atom over its two possible states, 

(,) - £ , V, = i [(He™ + (-1).—] - f^f = tanM^, . (« 

The next assumption is that all of the all atoms will behave like this one. Therefore, the average spin for 
the entire material is 

S = «*» - (',) , (9) 

which now allows us to write 

s — tanh/3Jns, (10) 

which is the final result for this section. This equation can then be used to solve for the Curie temperture, 
the total magnetic moment and the magnetic susceptibility. To be more general, an external magentic field 
can be applied as well which would alter the solution but not the form. For this case, the Hamiltonian would 
be, 

HieS = —JSiUSj - (J,BSi = -es l , (11) 

where /Lt is the magnetic moment, B is the magnetic field strength and e is the net effective energy on the i th 
atom. Note however that while this does not alter the solution using the mean field approximation becasuse 
this is the effective Hamiltonian (fTTj) for a single atom. In other words, the averaging is already taken into 
account in the effective Hamiltonian. If one were to write the full Hamiltonian with the external field instead 
of (JTJ above, then one will not have as simple of a solution for ([3]l. i.e. Z e g ^ (2 cosh/3e) N . 



3 Using ME for the approximation 

Part of the strength of using ME is that one does not need to justify some of the assumptions such as those 
used above (4 explicitly used, many hidden). One simply supplies the information that one has available 
and allows the method to turn out the most honest solution based on the information given. This can be 
demonstrated by using the method to find an appropriate approximation for a ferromagnetic material given 
the information that one has available. We show in this section where ME was used for this very same task 
to arrive at a similar solution as above. 

We start by writing the full posterior solution for a ferromagnetic system (all atoms) which is a canonical 
distribution, 

p{a} = I e -W.} j (12) 

where p{s} = p(s\ . . . sat), Z is the patition function from ^) with H {s} being some general Hamiltonian 
and N the number of atoms. NOTE: This function can be assertained by using the ME method without 
assumptions as Jaynes did for Gibbs' canonical function when he used his special case of ME, MaxEnt [2]. 
However, to determine the values of the Curie temperture, the total magnetic moment and the magnetic sus- 
ceptibility we are faced with the same dilema as above; we cannot compute the solutions exactly. Therefore, 
we must try to find an apporximation that is tractable. For illustrative purposes, we will adopt the Ising 
model so as to show how one can use ME to arrive at a similar approximation as the mean field produces. 
For this we let 

N,n N 

H {s} = H int + H ext = -J*^2 SiS J ~ X! ^ Bsi ' ( 13 ) 

i,j i 

where in Hi„ t = H from (TTJ) and H ext is the energy due to external magnetic field acting on the atoms. We 
now wish to use ME to find an approximation that is tractable. We accomplish this by first writing down 
the appropriate entropy, 

s[p A ip] = -y>Aiog^ , (u) 

where p is the canonical probability using (|13j) and pa is the approximation that we seek. We proceed 
rewiting the entropy as 

S\pa\p} = -^PA^og p A + ^2p A log p , (15) 
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or subsituting with the canonical form for p, (|12[) 



S[ PA \p] = \s a -Y,PaPH{s}-PF , (16) 



W 

where Z can also be written as Z = e~@ where F is the free energy. This can further be reduced to 



S\p A \p] = -S A +l3(F-(H{s}) A ) , (17) 



where (H {s}) A = J2{ s } PaH {s} which can also be seen as the energy E A of the system. Since by definition, 
5[pa|-P] — we can wr he, 

F < E A — TS A . (18) 

When using the ME method, we maximize the entropy to find the best posterior given the information 
provided. For this specific case, we have rewritten the entropy form in terms of an inequality that compares 
the free energy of the system, F with the approximate values for the average energy, E A and the entropy 
S A . The free energy is minimized to determine the best approximation for F, which is called the Bogoliubov 
Variational Principle. [5] ■ However, this is simply a special case of ME. To be more general, we proceed with 
using the ME method to find the best approximation. 

When using the ME method, the goal is to search the family of possible posteriors to find the one that 
maximizes the entropy given the constraints. In addition to this, wc all need a solution that is trackable. 
Therefore, we seek a posterior that has a form, 

PA = 7^e"^ A , (19) 
Z A 

where H A = — J2i e iASi and ti A is some effective energy similar to (fTTj) . Again for illustrative purposes we 
assume that all atoms poses the same effective energy so that e^A = £A- The difference is that we do not yet 
know the form of e A . We continue by following a similar route to ([8]) by writing the expection value for Si 
with respect to p A , 

( s i)A = PA = tanh^e,;A = tanh^eA , (20) 

{s,} 

Except that here we are marginalizing over all atoms except the i th . Notice that becasue e A is constant, the 
solution is independent of i. Next we maximize (|14[) or in keeping with the current case, we minimize the 
free energy, F. Rewiting (fT5|) 

(3F min = -jS A +P(H{ S }) A (21) 



Subsituting in (IT91 and rewritting, 



Fmi„= Nj$>Al0ge-^ A -i^p A l0gZ A +J2paH{s} , 

V W P M / W 



(22) 



as Z A is a constant it can be pulled out of the summation and since X){ s }Pa = T 

F min = -i 5>a/?#a - i logZ A )+J2paH {s} (23) 
\ P M P J M 



We now substitute our Hamiltonians into the function 



F min = - (H A ) A - - log Z A + (H {s}) A . (24) 



N,n N 



-e A ^2si\ - -log Z A + (-J^SiSj -^fiBsi) , (25) 



A \ i,3 1 1 A 
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and substituting (£\ Si) A = £\ (s») A = AT (sj) A = 7Vtanh/3eA yields, 



JV,n 

^min = £aN tanh /3e A - -3 log Z A - J X! ( S * S J') A ~ ^ Biv tanh £ e A ■ (26) 

Then substituting Z A « (2 cosh/JeA)^ yields, 

i^min = e A iVtanh/3e A - - log(2 cosh/^^ - J^Z ^' S ^A ~ M^iV tanh /3e A (27) 

i.j 

and since (|2"0|) is independent of i, (sj) A = (sj) A and (siSj) A — (sj) A (sj) A , we can write, 

Fmin = eA^tanh^eA - -^log(2cosh/3e A ) - J^I (s*) A («i) A ~ M-BiV tanh /3e A . (28) 

ij 

Substituting (|2"0)l into the equation yields, 

N 1 9 

fmin = e A Ntanh/3e A - — log(2 cosh /3e A ) - J -Nn (tank f3e A ) - ^B N tanh /3e A , (29) 
P ^ 

where the factor n comes from the number of nearest neighbors, the factor N is from the total number of 
atoms and the 1/2 is due to double counting. 

Now we choose the form that minimizes -F m in with respect to e A , 

dF min /3 N /3 sinhpe A f . 

— = = 7Vtanh/3e A + Ne A - T - — — 30 

oe A cosh fit A P cosh^e A 

-J-Nn2 tanh fJe A ^ uBN ^ 



2 cosh /3ea cosh /3eA 



or rewritten, 



' cosh 2 ^CA 



ATtanh/3e A + ATe A 4^ iVtanh/3e A (31) 



-JNn tanh /3e A 73 fiBN y 



After canceling terms we have, 
or better, 



cosh /3e A cosh (3e A 

= e A — /-t-B — Jn tanh /3ca (32) 
eA — f-iB = Jn tanh f3t A . (33) 



This is our final result for this section. Notice that if B = then we recover our solution using the mean 
field approximation above. If we let e A = Jns we rewrite the above equation, 

Jns = Jn tanh (3 Jns , (34) 

and after cancelations, 

s = tanh (3 Jns . (35) 

which is (fT0|) . 

However, from (|33j) alone we can solve for the Curie temperture, the total magnetic moment and the 
magnetic susceptibility in the usual way [7|. This is true even though we still do not know the form for e A . 
Using this approach, we did not need to assume that all atoms behave like a "central" atom and we did not 
need to know the explicit form of the effective energy, only that there was one. Following the ME method 
we simply processed all of our information that we had available. Notice that we also no longer need to 
assume Ising conditions. We explore such possible scenerios in the next section. 
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4 The defective Ising model 



4.1 Three state example 

In the Ising model, the possible states are spin up and spin down. Let us examine the case where there are 
three possible states. In this example, we will let s = {+1,0, —1} where an atom in the zero, state would 
contribute no energy, like perhaps a defect of some kind. We proceed with the method above up until (l20l) 
where the the new expected value for a particular atom for this case would be, 

/ \ 2sinh/3e A , . 

<-i>A = ^ « PA = 2cQsh/3eA + 1 • (36) 

{s,} 

Continuing the steps above, we write our new F m [ n , 

2sinh/3e A N 1 / 2sinh/3e A \ 2 2sinh/3e A , , 

^ = £AiV 2cosh/ A + 1 'J M2cosh^ A + 1) - J-Nn f ^-^ A + J - ^^^ ,(37) 

and minimize this function once again with respect to e A which yields, 

dF min 2sinh/3e A Af ^ /32cosh/3e A /34sinh 2 /3e A \ 2/3sinh/3e A 

v = Iv- r + Ae A ' 



<9e A 2cosh/3e A + l \2cosh/3e A + l (2 cosh/3e A + l) 2 / /3 2 cosh/3e A + 1 



jl Nn2 ( 2sinh/3e A \/ /32cosh/3e A /34sinh^e A \ fljy . / /32cosh/?e A /34sinh 2 /3e A 

2 ™ ^2cosh/3e A + ^2cosh/3e A + 1 (2 cosh^e A + l) 2 J ^ \2 cosh/3e A + 1 (2 cosh/3e A + l) 2 

After canceling terms we arrive at, 

/ 2sinh/3e A \ , N 

e A - fj,B = Jn , a ■ (40) 



2cosh^e A + 1 

Which is our final result. This equation can then be used as above to solve for the Curie temperture, the 
total magnetic moment and the magnetic susceptibility for this material. 
To make this result a little more genreral we can also write, 



e A -^B = Jn G (-^) = Jno? (g^) 



Jn G (( Si ) A ) , (41) 



where Z A is once again the approximate partition function, n G are the nearest neighbors and F A is the 
approximate free energy. 



4.2 Two energy example 

In the previous example, we examined a three state atom. In this example, we examine a material that has 
an known defect. Perhaps the material was scanned in some way and a defect was found. In this case, we 
need to use two effective energy terms. One for the atoms that are surounded by non-defective atoms, end 
and one for the ones affected by the defect, er>. For illustrative purposes, we will examine the case where we 
have one defective atom. This means that there are N — n — 1 atoms that are surrounded by non-defective 
atoms, n atoms that have one defective atom next to it and 1 atom which is the defective atom. For our 
purposes, let's think of the defect as an empty slot. Therefore, the actual Hamiltonian would be, 

AT— n— 1,73 ra, n— 1 N—n—1 n 

H {s} — Hi n t + H ext = — J ^2 s l Sj — J ^ s k si-Js - ^2 fJ.Bsi-^2fiBs k - fJ-Bso , 

i,j k.l i k 

where i are the non-defective atoms, j are neighbors (n) for these atoms, k are the atoms affected by the 
defect, I are the neighbors (n — 1) of the affected atoms and Sq is the defective atom. Since we are looking 
at this as an empty slot, we let sq = 0. Now we write our estimated Hamiltonian for this case, 

Ha = —£nd ^ s% — £d ^ «fe • 

i k 



G 



F min = - (H A ) A - - log Z A + (H {s}) A , (44) 



Following the same procedures above, we attain two expected values, one for each effective energy, 

(si) A = *^2si pa = t&nh[3e-Ni> (42) 

{»>} 

and 

( s fc)A = E s fc Pa = tanh/3e D . (43) 

Once again we write down the function we wish to minimize, 

1 

and substituting in our Hamiltonians yields, 

/ N-n-l " \ 1 / N—n—l,n n,n-l N-n-1 n \ 

F m in = —( -«ND S » " £ D X! Sfc / ~~ fl l0g Z ^ + \ ~ J E S ' iS J ~~ J E SfcS ' ~~ X! ^ BSi ~ X! ^' B ' S ' £ ) ' 

\ i k I A \ *>i W * fe ' A 

(45) 

Substituting Sj) A = J2i ( s »)a = (-W - ri - 1) (s») A = (N - n - 1) tanh/3e N D and (£) fc s fc ) A = J2k ( s *-')a 
ii (sfc) A = ntanh/3eD yields, 

i^min = £ nd {N-n-l) tanh /3e ND + e D n tanh /3e D - log (Z ND Z D ) (46) 

AT— n— l,n n,n— 1 

-J ^2 ( s i s j)^B^ J E ^ SfcS ')D - ^ B [ £ nd (-W - n - 1) tanh/3e ND + e D ntanh/3e D ] (47) 

i,j k,l 

where Z A = Znd^d- Substituting Z ND « (2 cosh^e N D) (Ar_n_1) and Z D « (2 cosh/3e D ) (n) yields, 
i^min = end (JV - n - 1) tanh pe ND + e D ntanh/3e D - - log(2 coshp^D)^"" 1 - i log(2 cosh/3e D )"(4fe) 

iV — tl— 1,71 n,n — 1 

_ ^ X! ( s ' s j)nd " J E ( S fc s /) D ^ A 4 - 8 [ £ nd (^V - " - l)tanh/3e ND + e D ntanh^e D ] . (49) 
i,i fe,i 

Since (j42|) and (|43| are independent of i and fc respectively, (sj) A = (sj-) A and (siSj) A — (si) A (sj) A , we can 
write, 

F min = e ND (^-n-l)tanh/3e ND + e D (n)tanh/3e D -ilog(2cosh/3e ND ) Ar -"" 1 -ilog(2 cosh /3e D ) Tl_1 (50) 

iV — n— l,n n,n— 1 

( s ;)nd ( s j')nd — J *> Sfc ^D ( s ')d — [ £ND (-^ — 71 — 1) tanh/3eND + £d (ti) tanh/3e($]l) 
Substituting (|4"2")l and (14"51) into the equation yields, 

i^min = £ nd (iV - ti - 1) tanh pe ND + e D (ti) tanh /3e D - i log(2 cosh /JeND)^-"- 1 - ^ log(2 cosh /Jeo)"- 1 (52) 

1 2 1 2 

— —J (N — n — 1) (n) tanh /3e^u — (n) (n — 1) tanh f3eu — fJ-B [end (N — n—1) tanh /Send + £d («■) tanh $63] 
and after collecting end and £d like terms we have, 

-Fmin = e ND (N -n- l)tanh/3e ND - i log (2 cosh^eND)^ - ™ -1 - - J (N — n- 1) (n) tanh 2 Pe ND ^iBe ND (N - n - l)tanh, 

P 2 

1 _ 1 „ 

+cd (ti) tanh/?£D — ^ log(2 cosh/3eD)" — - J (n) (n — 1) tanh /3ed — K/J-Bcd (ti) tanh /3cd , 
P 2 

where the factor n comes from the number of nearest neighbors, the factor N is from the total number of 
atoms and the 1/2 is due to double counting. 
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Now we choose the form that minimizes -F m i n with respect to cnd and Ed, 

5i?min n /at v. a | / u P N - n - 1 /3sinh/3e ND , . 

— = = (N - n - l)tanh/3e ND + [N - n - 1) e N D rTT, a ZTa ( 54 ) 

oend coslr/3e ND p cosh/3e ND 

- ji (N-n-1) (n) 2 tanh ^e ND ^ fiB(N-n-l) 



2 cosh /3end cosh /3end 



= (JV-n-l)tanh^eND + (JV-n-l)eKD — 5 Q (55) 



P N - n - 1 P sinh /3e ND 

' cosh 2 /3e N D cosl1 ^nd 



- J (AT - n - 1) (n) tanh /3e ND — ^ fiB(N-n- 1) 1- 



cosh /3end cosh /?£nd 

= end — J (n) tanh/3eND — A*-S 

£nd ^ M-B = tanh /3 end (56) 

cd — (J'B = J {n — 1) tanh/3eD 

Which is our final result. These equations can then be used as above to solve for the Curie temperture, 
the total magnetic moment and the magnetic susceptibility for this material as usual. The difference here 
is that we will have two of each. For example, a Curie temperture that applies to most of the atoms, and 
one that is local to the defect. Therefore, the total magnetic moment will not simple be a result of just one 
tempreture but a sum of each magnetic moment determined by the equations. Notice that (|41[) still holds in 
general. In the two energy case, for one energy the number of nearest neighbors, hq — n and for the second 
energy, ug = n — 1. 



5 Conclusions 

We described the traditional Ising model of a ferromagnetic material and the mean field approximation for 
a multidimentional material. It has been shown that not only can the Me method be used for updateing, 
but for determining approximations as well. 

Acknowledgements: I would like to acknowledge valuable discussions with Ariel Caticha regarding this 
material. 
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